Fusion of multiple imaging planes for isotropic imaging in MRI and quantitative image analysis using isotropic or near-isotropic imaging

ABSTRACT

In accordance with the present invention there is provided methods for generating an isotropic or near-isotropic three-dimensional images from two-dimensional images. In accordance with the present invention the method includes, obtaining a first image of a body part in a first plane, wherein the first image generates a first image data volume; obtaining a second image of the body part in a second plane, wherein the second image generates a second image data volume; and combining the first and second image data volumes to form a resultant image data volume, wherein the resultant image data volume is isotropic or near-isotropic.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.10/728,731, filed Dec. 4, 2003, entitled “Fusion of Multiple ImagingPlanes for Isotropic Imaging in MRI and Quantitative Image Analysisusing Isotropic or Near-isotropic Imaging,” which in turn claimspriority from U.S. Provisional Patent Application Ser. No. 60/431,176,filed Dec. 4, 2002. Each of the above-described applications is herebyincorporated herein by reference, in their entirety.

STATEMENT AS TO RIGHTS TO INVENTIONS MADE UNDER FEDERALLY SPONSOREDRESEARCH AND DEVELOPMENT

Certain aspects of the invention described below were made with UnitedStates Government support under Advanced Technology Program 70NANBOH3016awarded by the National Institute of Standards and Technology (NIST).The United States Government may have rights in certain of theseinventions.

TECHNICAL FIELD

This invention relates generally to medical imaging, and morespecifically to medical imaging that facilitates analysis in more thanone dimension, e.g. magnetic resonance imaging (MRI). More particularlythe invention relates to isotropic imaging techniques used in medicalimaging, such as MRI, to improve quantitative image analysis.

BACKGROUND OF THE INVENTION

Magnetic resonance imaging (MRI) is a noninvasive imaging technique thatprovides clinicians and diagnosticians with information about theanatomical structure and condition of a region of interest within asubject. See, for example, U.S. Pat. No. 5,671,741 to Lang et al. issuedSep. 30, 1997 for “Magnetic Resonance Imaging Technique for TissueCharacterization;” U.S. Pat. No. 6,219,571 B1 to Hargreaves et al.issued Apr. 17, 2002, for “Magnetic Resonance Imaging Using DrivenEquilibrium Fourier Transform;” U.S. Pat. No. 6,479,996 to Hoogeveen etal. issued Nov. 12, 2002 for “Magnetic Resonance Imaging of SeveralVolumes;” U.S. patent application Ser. No. 2002/0,087,274 A1 toAlexander et al. published Jul. 4, 2002 for “Assessing the Condition ofa Joint and Preventing Damage.” Commonly, in MRI, a substantiallyuniform temporally constant main magnetic field (B.sub.0) is set up inan examination region in which a subject being imaged or examined isplaced. Via radio frequency (RF) magnetic field (B.sub.1) excitation andmanipulations, selected magnetic dipoles in the subject that areotherwise aligned with the main magnetic field are tipped to excitemagnetic resonance. The resonance is typically manipulated to inducedetectable magnetic resonance echoes from a selected region of thesubject. In imaging, the echoes are spatially encoded via magneticgradients set up in the main magnetic field. The raw data from the MRIscanner is collected into a matrix, commonly known as k-space. Byemploying inverse Fourier, two-dimensional Fourier, three-dimensionalFourier, or other known transformations, an image representation of thesubject is reconstructed from the k-space data.

Conventional MRI scans produce a data volume, wherein the data volume iscomprised of voxels having three-dimensional characteristics. The voxeldimensions are determined by the physical characteristics of the MRImachine as well as user settings. Thus, the image resolution of eachvoxel will be limited in at least one dimension, wherein the loss ofresolution in at least one dimension may lead to three-dimensionalimaging problems.

There are many applications in which depth or three-dimensional (“3D”)information is useful for diagnosis and formulation of treatmentstrategies. For example, in imaging blood vessels, cross-sections merelyshow slices through vessels, making it difficult to diagnose stenosis orother abnormalities. Likewise, interventional imaging, such as needletracking, catheter tracking, and the like, requires 3D information.Also, depth information is useful in the so-called interactive imagingtechniques in which images are displayed in real or near-real time andin response to which the operator can adjust scanning parameters, suchas view angle, contrast parameters, field of view, position, flip angle,repetition time, and resolution.

Three-dimensional imaging generally involves either acquiring multipletwo-dimensional or slice images that are combined to produce avolumetric image or, alternately, the use of three-dimensional imagingtechniques. Much effort at improving the efficiency of volume imaginghas been focused on speeding up the acquisition. For example, manytwo-dimensional fast scan procedures have been adapted tothree-dimensional imaging. Likewise, efforts have been made to improvereconstruction speed and efficiency, for example, through the use ofimproved reconstruction algorithms. Nevertheless, three-dimensionalimaging remains relatively slow.

However, current MRI acquisition techniques do not provide highresolution in all planes and quantitative image analysis using isotropicor near-isotropic imaging. Accordingly, the present inventioncontemplates new and improved magnetic resonance imaging techniques.

An additional problem not addressed by current 3D MRI scanning methodsis the reduction of partial volume effects. Partial volume effects arecaused when a voxel falls within the boundary between two scannedobjects. For example, if a patient's knee is being sagittally scanned, avoxel may be orientated such that part of the voxel falls within thefemur and part falls within a space outside of the femur. MR imagingwill average the overall gray value over the entire voxel. The lower thescanning resolution the greater the partial volume effects. In a 3Dscan, where there is low resolution in at least one plane of the scanimpact of the partial volume effects is greatly increased. Thus, thereis a need for methods of forming 3D MRI scans with reduced impact ofpartial volume effects.

Still further, an additional shortcoming of conventional 3D MRI scanningprocedures is that boundaries of scanned objects may be missed due toscanning resolution and scan orientation. This may occur when a boundaryof an object being scanned lies between the slice thickness of the scanor the boundary of an object is parallel to the imaging plane. Thereforethere is a need for improved methods for reducing the likelihood ofmissed boundaries.

SUMMARY OF THE INVENTION

The invention addresses the problem that with current 3D imageacquisition techniques the in-plane (x-y plane) resolution of the slicesis usually at least 3 times higher than the slice thickness (inz-dimension). The low resolution between the slices (typically inz-direction) leads to limitations with respect to 3D image analysis andvisualization. The structure of 3-dimensional objects cannot bedescribed with the same accuracy in all three dimensions. Partial volumeeffects affect interpretation and measurements in the z-dimension to agreater extent than in the x-y plane. Thus, resolution and accuracy ofmultiplanar reformations depend on the slicing direction through thevolumetric data.

In addition, the invention also addresses the issue of increasingaccuracy of tissue segmentation and/or quantitative analysis of images,such as MR images. For example, after obtaining an isotropic ornear-isotropic three-dimensional MR image (e.g., using pulse sequenceacquisition techniques described herein and known in the field),particular tissues can be extracted from the image with greater accuracyand, moreover are quantitative. Currently available subjective visualinspection techniques are not quantitative and, additionally, are ofteninaccurate.

Thus, in one aspect, a method of improving resolution of images, such asMR images, is provided. In certain embodiments, the method includes, forexample, obtaining at least two MR scans (e.g., scans in perpendicularplanes) of a body part and merging the scans, thereby increasingresolution. In any of the methods described herein, the scans may be inany plane, for example, sagittal, coronal and/or axial imaging planes.Preferably, the second or subsequent scans contain a sufficient numberof slices to cover the entire field of view of the first scan.Furthermore, in any of the methods described herein, the data obtainedfrom the two or more scans are subsequently merged to form a new datavolume, which is isotropic (or near-isotropic) and has a resolutioncorresponding to the in-plane resolution of S1 and S2. Merging mayinclude, for example, determining a gray value for each voxel (V) of thenew (merged) data volume. In certain embodiments, the gray values areobtained by: (a) determining the position in 3D space for V; (b)obtaining (e.g., from the original scans) gray values of the scans priorto fusion at this position; (c) interpolating (combining) gray valuesfrom S1 and S2 into a single gray value (G); and (d) assigning G to V.

In any of the methods described herein, any living tissue can be imaged,including, but not limited to, joints, bones and/or organs (e.g., brain,liver, kidney, heart, blood vessels, GI tract, etc.).

In accordance with the present invention there is provided a MRIscanning method, the method comprising, performing a first MRI scan of abody part in a first plane, wherein the first MRI scan generates a firstimage data volume; performing a second MRI scan of the body part in asecond plane, wherein the second MRI scan generates a second image datavolume; and combining the first and second image data volumes to form aresultant image data volume, wherein the resultant image data volume isisotropic.

In accordance with another embodiment of the present invention there isprovided a method for producing isotropic or near-isotropic image data,the method comprising: obtaining a first image data volume from a firstMRI scan in a first plane; obtaining a second image data volume from asecond MRI scan in a second plane; extracting boundary image data fromeach of the first and second image data volumes; combining saidextracted boundary image data to form a resultant image data volume.

In accordance with the present invention there is provided a method forgenerating a three dimensional data volume, the method comprising:acquiring at least two data volumes from at least two MRI scansperformed in two different planes; combining the data volumes to form aresultant data volume; selecting a therapy in response to the resultantdata volume; and deriving a shape for an implant.

The system includes an image analysis method. The image analysis isperformed by obtaining a first image of a body part in a first plane,wherein the first image generates a first image data volume; obtaining asecond image of the body part in a second plane, wherein the secondimage generates a second image data volume; and combining the first andsecond image data volumes to form a resultant image data volume, whereinthe resultant image data volume is isotropic. Additionally, first andsecond gray values can be obtained from the first and second image datavolumes at one or more three-dimensional positions. That data can thenbe interpolated to provide a resultant gray value which is then assignedto a voxel in the three-dimensional position of the resultant datavolume. As will be appreciated by those of skill in the art, the anglebetween the images can range from about 0° to 180°, or from 0° to 90°.Once these values have been obtained, a therapy or treatment can beselected to complement the data volume. A person of skill in the artwill appreciate that at least one additional image of a body part takenin a plane different than any previous plane used can be taken used togenerate additional image volume. From that image volume, data volume isgenerated which can then be combined with the first and second imagedata volumes to form a resultant data volume. Of course, extracting aboundary image data volume from the resulting image data volume can alsobe performed, if desired.

A method is also described for producing isotropic or near-isotropicimage data from images. This method generally comprises: obtaining afirst image data volume from a first image in a first plane; obtaining asecond image data volume from a second image in a second plane;extracting boundary image data from each of the first and second imagedata volumes; and combining the extracted boundary image data to form aresultant image data volume. Of course, it is possible to also obtainingat least one additional image data volume from at least one additionalimage in a plane different than the first plane and the second plane;extracting an additional boundary image data from the additional imagedata volume; and combining the additional boundary image data volumewith the resultant image data volume. This resultant data can beisotropic or near-isotropic. As will be appreciated, the first plane canbe at an angle relative to the second plane; that angle can be fromabout 0° to 180° or from about 0° to 90°.

A method is included for generating a three dimensional data volume.Generally, this method includes acquiring at least two data volumes fromat least two images performed in two different planes; combining thedata volumes to form a resultant data volume; selecting a therapy inresponse to the resultant data volume; selecting an implant; andderiving a shape for an implant. The combining step can further includeobtaining gray values for each data point in each of the data volumes;interpolating a resultant gray value from gray values; and assigning theresultant value to each data point of the resultant data volume. Priorto combining the data, data corresponding to any surface can be scannedin each plane and extracted.

Another method for generating three dimensional data is also disclosed.This method includes obtaining a first image in a first plane producinga first data volume with a default resolution; obtaining a second imagein a second plane producing a second data volume with the defaultresolution; combining the first and second data volumes to produce aresultant data volume, the resultant data volume having a resultantresolution. As will be appreciated, the resultant resolution is greaterthan the default resolution.

An image analysis method is disclosed that includes the steps ofobtaining at least one image of a body part in at least a first planeand a second plane, wherein the first plane generates a first image datavolume and the second plane generates a second image data volume; andcombining the first and second image data volumes to form a resultantimage data volume, wherein the resultant image data volume is isotropic.

An alternative image analysis method is also disclosed that includesobtaining at least one image of a body part in at least a first planeand a second plane, wherein the first plane generates a first image datavolume and the second plane generates a second image data volume; andcombining the first and second image data volumes to form a resultantimage data volume, wherein the resultant image data volume isnear-isotropic.

In accordance with the present invention there is provided a method forgenerating three dimensional MRI scan data, the method comprising:performing a first MRI scan in a first plane producing a first datavolume with a default resolution; performing a second MRI scan in asecond plane producing a second data volume with the default resolution;combining the first and second data volumes to produce a resultant datavolume, the resultant data volume having a resultant resolution.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates two MRI scans illustrating data volumes S1 and S2;each of the scans shows a plurality of image slides taken in planesparallel to the initial scan.

FIG. 2 illustrates a set of three voxels produced by an image scanillustrating an increased z-axis length.

FIG. 3 illustrates a first set of three voxels produced by an image scanillustrating a z-axis component.

FIG. 4 illustrates a second set of three voxels produced by an imagescan illustrating a z-axis component.

FIG. 5 illustrates a resultant set of nine voxels generated by themethods in accordance with the present invention.

FIG. 6 illustrates a combined boundary image data extracted from twoimage scans.

FIG. 7 illustrates a three-dimensional implant design generated from atleast two image scans.

FIGS. 8A-C illustrate flow charts illustrating processes of theinvention.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

The following description is presented to enable any person skilled inthe art to make and use the invention. Various modifications to theembodiments described will be readily apparent to those skilled in theart, and the generic principles defined herein can be applied to otherembodiments and applications without departing from the spirit and scopeof the present invention as defined by the appended claims. Thus, thepresent invention is not intended to be limited to the embodimentsshown, but is to be accorded the widest scope consistent with theprinciples and features disclosed herein. To the extent necessary toachieve a complete understanding of the invention disclosed, thespecification and drawings of all issued patents, patent publications,and patent applications cited in this application are incorporatedherein by reference.

As will be appreciated by those of skill in the art, methods recitedherein may be carried out in any order of the recited events which islogically possible, as well as the recited order of events. Furthermore,where a range of values is provided, it is understood that everyintervening value, between the upper and lower limit of that range andany other stated or intervening value in that stated range isencompassed within the invention. Also, it is contemplated that anyoptional feature of the inventive variations described may be set forthand claimed independently, or in combination with any one or more of thefeatures described herein.

The present invention is of a method of image analysis that can be usedfor improving tissue segmentation and/or quantifying image analysis.Specifically, the present invention combines two or more images toachieve high resolution in all three-dimensional directions. Theprinciples and operation of the method according to the presentinvention may be better understood with reference to the accompanyingdescriptions.

1.0 General Overview

According to the present invention, a method of improving resolutionand/or tissue segmentation of images taken of a body part is described.This method typically involves acquiring at least two images indifferent planes and combining the images to achieve the same (e.g.,high) degree of resolution in all directions. The images can beacquired, for example, by using an MRI. However, as other imagingdevices become available, those of skill in the art will appreciate thatthese techniques can be provided to other imaging devices as well,without departing from the scope of the invention.

The methods described herein provide isotropic or near-isotropicresolution which results in improved tissue segmentation. Unlikecurrently employed visual inspection, which is highly subjective, themethods and compositions described herein are quantitative and,accordingly, increase the accuracy of diagnosis and design of treatmentregimes.

1.1 Magnetic Resonance Imaging (MRI)

Describing MRI in general terms, all protons within living tissues havean inherent magnetic moment and spin randomly giving rise to no netmagnetization or direction. When a specimen is placed within themagnetic field of the MR scanner, the protons continue to spin but alignthemselves parallel or anti-parallel to the direction of the field (B₀)corresponding to low and high-energy states respectively. In the courseof an MR examination, a radiofrequency (RF) pulse (B₁) is applied to thesample from a transmitter coil orientated perpendicular to B₀ and theprotons are momentarily tilted out of alignment; the precession of theinduced net transverse magnetization around the axis of the static B₀field produces a voltage across the ends of the receiver coil which isdetected as the MR signal. For a general discussion of the basic MRIprinciples and techniques, see MRI Basic Principles and Applications,Second Edition, Mark A. Brown and Richard C. Semelka, Wiley-Liss, Inc.(1999); see, also, U.S. Pat. No. 6,219,571 to Hargreaves, et al.

1.1 High Resolution 3D MRI Pulse Sequences

MRI employs pulse sequences that allow for better contrast of differentparts of the area being imaged. Different pulse sequences are bettersuited for visualization of different anatomic areas. More than onepulse sequence can be employed at the same time. A brief discussion ofdifferent types of pulse sequences is provided in International PatentPublication WO 02/22014 to Alexander et al. published Mar. 21, 2002.

Routine MRI pulse sequences are available for imaging tissue, such ascartilage, include conventional T1 and T2-weighted spin-echo imaging,gradient recalled echo (GRE) imaging, magnetization transfer contrast(MTC) imaging, fast spin-echo (FSE) imaging, contrast enhanced imaging,rapid acquisition relaxation enhancement, (RARE) imaging, gradient echoacquisition in the steady state, (GRASS), and driven equilibrium Fouriertransform (DEFT) imaging. As these imaging techniques are well known toone of skill in the art, e.g. someone having an advanced degree inimaging technology, each is discussed only generally hereinafter.

1.2. Measurement of T1 and T2 Relaxation

As a result of random thermal motion, the proton spins within a samplelose coherence with one another. This loss of coherence results insignal decay. The time taken for the MR signal to return to zero dependson many factors, one is the rate at which the energized spins loseexcess energy relative to their immediate environment. This phenomenoncalled spin-lattice, or T1 relaxation, affects mainly magnetizationparallel to B₀ and leads to a net loss of energy from the spin system.

Another phenomenon that is observed is that the spins of neighboringprotons tend to drift out of alignment with one another as a result ofslight differences in frequency. This causes a loss in phase coherence,referred to as spin-spin or T2 relaxation. T2 relaxation affects thetransverse component of the magnetization but does not cause a net lossof energy.

Conventional T1 and T2-weighted MRI depict living tissue such asarticular cartilage, and can demonstrate defects and gross morphologicchanges. One of skill in the art could readily select a T1 orT2-weighted MRI depending on the structure to be imaged. For example,T1-weighted images show excellent intra-substance anatomic detail ofcertain tissue such as hyaline cartilage while T2-weighted imagingprovides a better depiction of joint effusions and thus surfacecartilage abnormalities.

1.3 Gradient-Recalled Echo (GRE) Imaging

Gradient-recalled echo (GRE) imaging has 3D capability and the abilityto provide high resolution images with relatively short scan times. Fatsuppressed 3D spoiled gradient echo (FS-3D-SPGR) imaging has been shownto be more sensitive than standard MR imaging for the detection ofhyaline cartilage defects such as those typically occurring in the knee.

1.4 Magnetization Transfer Contrast Imaging

Magnetization transfer imaging can be used to separate articularcartilage from adjacent joint fluid and inflamed synovium.

1.5 Fast Spin-Echo (FSE) Imaging

Fast spin-echo (FSE) imaging is another useful pulse sequence MRItechnique. Incidental magnetization transfer contrast contributes to thesignal characteristics of on fast spin-echo images and can enhance thecontrast between tissues. Sensitivity and specificity of fast spin-echoimaging have been reported to be 87% and 94% in a study witharthroscopic correlation.

1.6 Echo Planar Imaging (EPI)

Echo planar imaging (EPI) is an imaging technique in which a series ofechoes is rapidly induced following a single radiofrequency (RF) pulse.More specifically, an RF pulse and a slice select gradient are appliedto excite resonance in a selected slice and a phase encode gradient isapplied to phase encode the resonance. A series of frequency encode orread gradients of alternating polarity is applied in successive fashion.During each read gradient, a magnetic resonance signal or echo is readout. Between each read gradient, a short pulse or blip along the phaseencode gradient axis is applied to increment the phase encoding of theresonance by a line in the selected slice. A one-dimensional inverseFourier transform of each echo provides a projection of the spindistribution along the read axis. A second inverse Fourier transformalong the phase encoded echoes provides a second dimension of spatialencoding. Typically, the phase encode gradient blips are selected of anappropriate magnitude that data for a complete field of view is takenfollowing each RF pulse. The total sampling time is determined by thenumber of sampled points per read gradient and the number of phaseencode gradient steps.

Echo volume imaging extends echo planar imaging techniques to multipleplanes. After performing the above-described echo planar imagingsequence, a pulse or blip along a secondary phase encoding axis isapplied. Typically, the secondary phase encoding blips step the phaseencoding along an axis perpendicular to the primary phase encode andread axes. Thereafter, phase encode gradient blips are applied betweeneach read gradient to step line by line in the primary phase encodedirection. Because the phase encode blips in the first k-space planemove the phase encoding to one extreme edge of the field of view, thephase encoding blips in the second k-space plane in the secondary phaseencode direction are typically of the opposite polarity to step thephase encoding back in the opposite direction. In this manner, themultiple planes are aligned, but offset in steps in the z-direction. Onedisadvantage of the above echo planar imaging and echo volume imagingtechniques is that the trajectory through k-space is reversed in timefor alternate phase encode lines or views. This causes phasediscontinuities that can result in ghosting.

Spiral echo planar imaging techniques are also known, in which theapplied x- and y-gradient pulses, i.e., along the traditional read andphase encode axes, are sinusoidally varying and linearly increasing. Inthis manner, data sampling commences at the center of the field of viewand spirals outward, covering the field of view along a spiral k-spacetrajectory. One of the drawbacks of spiral echo planar imaging, however,is that it is a single slice technique. To obtain multiple slices, thespiral echo planar imaging technique is repeated multiple times. An RFexcitation pulse and slice select gradient followed by sinusoidallyvarying and linearly increasing x and y-gradients are applied for eachslice to achieve coverage of the volume of interest.

1.7 Contrast Enhancing Imaging

The use of gadolinium in imaging has been applied in several differentforms. For example, direct magnetic resonance (MR) arthrography, whereina dilute solution containing gadolinium is injected directly into atissue (e.g., joint), improves contrast between cartilage and thearthrographic fluid. Indirect MR arthrography, with a less invasiveintravenous injection, can also been applied. Gadolinium enhancedimaging has the potential to monitor glycosaminoglycan content, whichmay have implications for longitudinal evaluations of injured softtissue such as cartilage.

1.8 Driven Equilibrium Fourier Transformation

Another 3D imaging method that has been developed is based on the drivenequilibrium Fourier transform (DEFT) pulse sequence (U.S. Pat. No.5,671,741 to Lang et al. issued Sep. 30, 1997), and may be specificallyutilized for soft tissue (e.g., cartilage) imaging. DEFT provides aneffective tradeoff between T2/T1 weighting and spin density contrastthat delineates the structures of interest. Contrast-to-noise ratio may,in certain tissues/structures, be greater with DEFT than with spoiledgradient echo (SPGR). DEFT is an alternative approach to SPGR. DEFTcontrast is very well suited to imaging articular cartilage. Synovialfluid is high in signal intensity, and articular cartilage intermediatein signal intensity. Bone is dark, and lipids are suppressed using a fatsaturation pulse.

1.9 A Representative Example of MR Imaging

A MR image can be performed using a whole body magnet operating at afield strength of 1.5 T (GE Sigma, for example, equipped with the GESR-120 high speed gradients [2.2 Gauss/cm in 184 μsec risetimes]). Priorto MR imaging, external markers filled with Gd-DTPA (Magnevist®, BerlexInc., Wayne, N.J.) doped water (T1 relaxation time approximately 1.0sec) can be applied to the skin. External markers can be included in thefield of view of all imaging studies. Patients can be placed in thescanner in supine position and the appropriate area imaged. After anaxial scout sequence, coronal and sagittal T1-weighted images can beacquired using the body coil (spin-echo, TR=500 msec, TE=15 msec, 1excitation (NEX), matrix 256>128 elements, field of view (FOV) 48 cm,slice thickness 7 mm, interslice spacing 1 mm). The scanner table canthen be moved to obtain coronal and sagittal images using the samesequence parameters. These T1-weighted scans can be employed to identifyaxes that can be used later for defining the geometry of the tissue. Arapid scout scan can be acquired in the axial plane using a gradientecho sequence (GRASS, 2D Fourier Transform (2DFT), TR=50 msec, TE=10msec, flip angle 40°, 1 excitation (NEX), matrix 256×128 elements, fieldof view (FOV) 24 cm, slice thickness 7 mm, interslice spacing 3 mm).This scout scan can be used to determine all subsequent high resolutionimaging sequences centered over the body part. Additionally, using thegraphic, image based sequence prescription mode provided with thescanner software, the scout scan can help to ensure that all externalmarkers are included in the field of view of the high resolution MRsequences.

There are several issues to consider in obtaining a good image. Oneissue is good contrast between different tissues in the imaged area inorder to facilitate the delineation and segmentation of the data sets.In addition, if there are external markers, these must be visualized.One way to address these issues is to use a three-dimensional spoiledgradient-echo sequence in the sagittal plane with the followingparameters (SPGR, 3DFT, fat-saturated, TR=60 msec, TE=5 msec, flip angle40°, 1 excitation (NEX), matrix 256×160 elements, rectangular FOV 16×12cm, slice thickness 1.3 mm, 128 slices, acquisition time approximately15 min). Using these parameters, one can obtain complete coverage acrossthe body area and the external markers both in mediolateral andanteroposterior direction while achieving good spatial resolution andcontrast-to-noise ratios. The fat-saturated 3D SPGR sequences can beused for rendering many tissues in three dimensions, e.g. cartilage. The3D SPGR sequence can then be repeated in the sagittal plane without fatsaturation using the identical parameters and slice coordinates usedduring the previous acquisition with fat saturation. The resultantnon-fat-saturated 3D SPGR images demonstrate good contrast between lowsignal intensity cortical bone and high signal intensity bone marrowthereby facilitating 3D rendering of the femoral and tibial bonecontours. It is to be understood that this approach is representativeonly for joints and should not be viewed as limiting in any way.

1.10 Magnetic Resonance Imaging-Vertically Open Magnet (0.5 T)

MR imaging can also be performed using a 0.5 T vertically open MR unit(GE Signa SP, General Electric, Milwaukee, Wis.) and a MR trackingsystem. Prior to MR imaging, external markers filled with Gd-DTPA(Magnevist®, Berlex Inc., Wayne, N.J.) doped water (T1 relaxation timeapproximately 1.0 sec) can be applied to the skin. The subject can beplaced in upright position inside the magnet. The body part can beperpendicular to the main magnetic field. A 2DFT fast spin echo pulsesequence can be acquired in the sagittal plane (FSE, TR=4000 msec, TE=25msec, bandwidth 7.8 kHz, echo train length 8, 3 excitations, slicethickness 4 mm, interslice spacing 0.5 mm, matrix 256×192 elements,field of view 24 cm). For rapid scan acquisition with scan planetracking, a fast single slice gradient-echo pulse sequence can beacquired in the sagittal plane or in the axial plane (GRASS, TR=14 msec,TE=5 msec, flip angle 40 degrees, bandwidth 32 kHz, 1 excitation, slicethickness 4 mm, matrix 256×128 elements, field of view 20 cm, temporalresolution 2 sec/image). A field of view of 20 cm can be chosen in orderto achieve sufficient anatomic coverage in superoinferior.

2.0 Fusing Images

Despite the existence of these imaging techniques, resolution in morethan one plane remains difficult. In accordance with the presentinvention there is provided methods to overcome resolution difficultieswherein at least two data volumes from two separate images, such as MRIscans, are combined to form a single data volume having isotropic ornear-isotropic resolution.

Referring now to FIG. 1, there is shown at least two exemplary datavolumes S1 100 and S2 200 generated by at least two separate images. Asillustrated here, each data volume 100, 200 has a plurality of datavolumes 100 _((1-n)), 200 _((1-n)), as shown by the stacking of theimage slices. S1 is an image of a knee joint taken in the coronal plane,while S2 is an image of a knee joint taken in the sagittal plane. Inthis example, S1 and S2 are taken in planes that are perpendicular toeach other. However, as will be appreciated by those of skill in theart, other orientations and plane relationships can be used withoutdeparting from the scope of the invention.

Each data volume can have equal imaging dimensions in two dimensions,for example, the x and y-axes, while the imaging dimension in a thirddimension, e.g. z-axis, is greater than those in the first twodimensions, in this case the x-axis and y-axes. In a preferredembodiment a second scan can be taken at an angle between, for example,about 0° and 180° and more preferably between about 0° and 90°

Although the present invention is described using at least two scans, aperson of skill in the art will appreciate that more scans can be usedwithout departing from the scope of the invention. Thus, additionalscans in the same or other planes or directions can also be obtained andanalyzed. For example, if the first scan is acquired in the sagittaldirection, a second scan in the coronal or axial imaging plane can thenbe acquired.

It is possible that the second scan would have the same in-planeresolution as the first scan. The second scan could then contain asufficient number of slices to cover the entire field of view of thefirst scan, thereby resulting in two data volumes with information fromthe same 3D space.

As described above, the data volumes generated from each image includex, y, and z-axis coordinate data, wherein as shown in FIG. 1 the x andy-axis data is isotropic while the z-axis data is non-isotropic. Thisphenomenon is better shown in FIG. 2 where there is shown an exemplaryset of three voxels 200 as produced by an MRI scan in accordance withthe present invention. The voxels 200 shown in FIG. 2 are shown beingorientated in the z-axis wherein the arrow 210 indicated the slicethickness of the image, in this case an MRI scan. The voxels 200 furtherinclude a physical item 220 to be imaged. As shown in FIG. 2 it can beseen that due to the slice thickness 210, information pertaining to thephysical item 220 to be imaged results in decreased accuracy.

In addition to potentially missing data another problem with the largerslice thickness is the increase of partial volume effects. A partialvolume effect occurs when a voxel only covers part of an object to beimaged, thus the gray value of the voxel is averaged instead of being atrue gray value. As shown in FIG. 2, a partial volume effect occurs whena pixel or voxel is partially disposed over an object to be imaged 220.Since the voxel 200 is disposed partially over the object to be imagedthe voxel's gray value will be averaged. To reduce the occurrence ofpartial volume effects, the present invention reduces the slicethickness of the scan, thereby reducing the likelihood of each voxelfrom being partially disposed on the object to be imaged.

Referring now to FIGS. 3-5, there is shown an exemplary embodiment ofproducing an isotropic or near-isotropic voxel in accordance with thepresent invention. As shown in FIG. 3, there is shown a set of threevoxels 300 produced by an image scan, wherein the voxels are shownhaving a z-axis 310 of greater length than the x-axis 315 and y-axis320.

Referring now to FIG. 4, there is shown a second set of three voxels 330produced by a second image scan, wherein the second scan was taken at anangle .theta. relative to the first scan. As discussed above, .theta.can range from, for example, about 0° to about 180°. As shown in FIG. 4,the second set of voxels has a z-axis dimension 340 greater than itsx-axis 345 and y-axis 350, wherein the z-axis 340 of the second set ofvoxels can be orientated in a plane different than that of the first setof voxels.

Referring now to FIG. 5, there is shown a third set of voxels 360consisting of nine voxels, wherein the third set of voxels 360 has beenformed by combining the first and second sets of voxels, wherein thez-axis data of the first set has been combined with x-axis or y-axisdata from the second set of voxels to form a new z-axis 370 of the thirdset of voxels 360, wherein the z-axis of the third voxel 370 has alength equal to or nearly equal to that of the x-axis 375 and y-axis380, therefore producing voxels having isotropic or near-isotropicdimensions.

After having performed at least two scans producing two data volume setsas shown in FIG. 1, the two data volumes are subsequently merged into athird data volume as shown in FIGS. 3-5. This resultant data volume isisotropic or near-isotropic with a resolution corresponding to thein-plane resolution of S1 and S2. The gray value for each voxel of thethird data volume is preferably calculated as follows: (a) determine theposition in 3D space for each voxel; (b) determine (e.g., look up) thegray values in S1 and S2 at this position; (c) employ an appropriateinterpolation scheme to combine the two gray values into a single grayvalue; and (d) assign each determined gray value to each voxel in theresultant data volume.

These manipulations may be repeated for more scans. Furthermore, tocompensate for differences in positioning between S1 and S2 of thescanned subject, e.g. due to motion, a registration technique such asprincipal-axis or volume-based matching can be applied.

3.0 Tissue Segmentation

In accordance with an alternative embodiment of the present inventionthere is provided a method of producing isotropic or near-isotropic MRIscan data from at least two image scans.

As described in detail above, two individual data volumes are obtainedfrom two separate image scans, wherein each of the scans have been takenat an angle .theta. relative to each other. In a preferred embodiment,the second scan, or image, is preferably taken at an angle .theta.between about 0° and 180° more preferably between an angle between about0° and 90°. Wherein each of the image scans produce individual datavolumes having x, y, and z components wherein the x and y components areisotropic or nearly isotropic and the z-axis size is determined by theslice thickness (or step length) of, for example, the MRI machine.

Tissue segmentation means can be applied to extract one or more tissuesfrom one or more images. This can be achieved with classification ofpixels or voxels of an electronic anatomical image (e.g. x-ray, CT,spiral CT, MRI) into distinct groups, where each group represents atissue or anatomical structure or combination of tissues or anatomicalstructures or image background. For example, as described above, everydata point of the first and second data volumes were combined to form aresultant data volume. Segmentation can then be performed on the entiredata volume or subportions of the data volume. While effective inproducing an isotropic or near-isotropic resultant data volume, theamount of data processing is great. Therefore, the method above requiresa fairly substantial amount of computer processing power as well as timeto complete the mathematical calculations required.

Referring now to FIG. 6 there is shown an exemplary embodiment of athree-dimensional MRI scan produced in accordance with the tissuesegmentation methods of the present invention, wherein data pertainingto the object to be imaged has been first extracted from each of thedata volumes prior to combining the extracted data to produce thethree-dimensional scan of FIG. 6. As shown in FIG. 6, the cartilagesurface of the medial femoral condyle is shown, wherein a sagittal scan400 and a coronal scan 450 were both acquired at a resolution of 0.27mm×0.27 mm in-plane resolution, and a 3 mm slice thickness with a 0.5 mmspacing. As shown in FIG. 6, the medial edge is outlined well in thecoronal scan, while the posterior edge of the condyle can be best seenin the sagittal scan.

In accordance with the alternative embodiment, data pertaining to asurface or an area of interest is first extracted from each data volumeproduced by two or more image scans. After extracting the data volumesof interest, each pixel or voxel in the data volumes are subsequentlymerged into a new data volume by transforming them into a commoncoordinate system. This can be achieved by explicitly computing atransformation matrix for one or more of the data sets through the useof anatomical or other user defined landmarks or through a prioriknowledge of the image position and orientation, such as the informationprovided by the DICOM imaging standard. To define a transformation in 3Dspace, the coordinates of four points in the original data volume andits corresponding location in the new data volume needs to beidentified. These coordinate pairs are used to set-up a linear system ofthe form:A×T=B,where A is the matrix with the original coordinates, T is thetransformation matrix and B is the matrix with the new coordinates. Thesolution to the above system is given by:T=B×A ⁻¹

Alternatively, a transformation matrix can be implicitly calculated byperforming a surface registration between the data sets. A surfaceregistration algorithm merges the two data volumes by minimizing a costfunction, such as a Euclidean distance transform and thus combining thevolume data. FIG. 6 shows an example of a resultant data volume. Thisresultant data volume is isotropic or near-isotropic with a resolutioncorresponding to the in-plane resolution of S1 and S2.

In another embodiment of the invention, a 3D MRI image is obtained usingany suitable technique, for example using pulse sequence acquisitionparameters that provide a 3D rather than a 2D Fourier Transformacquisition with isotropic or near-isotropic resolution, or by usingfusion of two or more 2D acquisitions. As used herein, isotropicresolution refers to an MRI image in which the slice thickness is equalto the in-plane resolution. Similarly, the term “near-isotropicresolution” refers to an image in which the slice thickness does notexceed more than 2× the in-plane resolution, more preferably not morethan 1.5× the in-plane resolution and even more preferably not more than1.25× the in-plane resolution. The isotropic or near-isotropic 3DFTimaging pulse sequence has advantages with regard to partial volumeaveraging. Partial volume averaging is typically not greater in slicedirection (z-direction) than in the imaging plane (x and y-direction).

Non-limiting examples of pulse sequences suitable for obtainingnear-isotropic or isotropic images include 3D FSE, 3D MFAST/3D SS-SPGR,3D FIESTA/3D SSFP, 3D FEMR, 3D DESS, 3D VIBE, and 3D SSFP. The preferredin-plane resolution of the 3DFT isotropic or near-isotropic imagingsequence is less than 0.5 mm and the preferred slice thickness is lessthan 0.8 mm, preferably less than 0.5 mm.

Subsequently, these isotropic or near-isotropic resolution images areused to increase the accuracy of segmentation or tissue extraction andany subsequent visualizations and/or quantitative measurements of thebody part, (e.g., measurement of cartilage thickness or size ofcartilage defects).

Thus, the invention described herein allows, among other things, forincreased resolution and efficiency of tissue segmentation or tissueextraction. Following the manipulations described herein (e.g., mergingof multiple images, isotropic or near-isotropic resolution imaging),commercially available segmentation software can be used, for examplesoftware that includes seed-growing algorithms and active-contouralgorithms that are run on standard personal computers.

For example, articular cartilage shown in the 3D MR images may beanalyzed. A sharp interface is present between the high signal intensitybone marrow and the low signal intensity cortical bone therebyfacilitating seed growing.

One exemplary, but not limiting, approach uses a 3D surface detectiontechnique that is based on a 2D edge detector (Wang-Binford) that hasbeen extended to 3D. This surface detection technique can generatesurface points and their corresponding surface normal. To smooth thecontour, the program samples 25 percent of the surface points and fits acubic spline to the sample points. The program can compute the curvaturealong sample spline points and find two sample points that have themaximum curvature and are separated by about half the number of voxelson the contour. These points partition the spline into two subcontours.For each subcontour, the program can compute the average distancebetween the points and the center of the mass.

Programs can allow the user, through the use of the mouse and/orkeyboard, the ability to observe the scene from arbitrary angles; tostart and stop the animation derived from the 3D data. Additionally, theuser can derive quantitative information on the scene through selectingpoints with the mouse.

The software programs can be written in the C++ computer language andcan be compiled to run, for example on Silicon Graphics Workstations orWindows/Intel personal computers.

4.0 Three-Dimensional Images

After the 3D MR image is obtained, by either using a 3D acquisition orby fusing two or more 2D scans as described above, and after one or moreanatomical objects have been extracted using segmentation techniques,for example, the object information can be transformed to a surfacerepresentation using a computer program. The program can, for example,be developed in AVS Express (Advanced Visual Systems, Inc., Waltham,Mass.). Every voxel has a value of zero if it is not within an object ofinterest or a value ranging from one to 4095, depending on the signalintensity as recorded by the 1.5 T MR. An isosurface can then becalculated that corresponds to the boundary elements of the volume ofinterest. A tessellation of this isosurface is calculated, along withthe outward pointing normal of each polygon of the tessellation. Thesepolygons can be written to a file in a standard graphics format (e.g.Virtual Reality Modeling Language Version 1.0: VRML output language) andvisualized on a computer screen.

Visualization programs are also available, for example, usercontrollable 3D visual analysis tools. These programs read in a scene,which scene consists of the various 3D geometric representations or“actors.” The program allows the user, through the use of the mouseand/or keyboard, the ability to observe the scene from arbitrary angles;to start and stop the animation derived from the 3D data. Additionally,the user may derive quantitative information on the scene throughselecting points with the mouse.

The software programs can be written in the C++ computer language and becompiled to run, for example, on Silicon Graphics Workstations andWindows/Intel personal computers. Biochemical constituents, for exampleof cartilage, may also be visualized, for example as described in WO02/22014 to Alexander.

A method is also described for producing isotropic or near-isotropicimage data from images. This method is shown in FIG. 8A. The first stepis to obtain an image 800. As shown by optional repeat step 801, thisstep can be repeated such that multiple images are obtained. Suitableimages include, for example, MRI. Once the image or images have beenobtained image data volume is obtained 810. Image data volume can beobtained one or more times as indicated by the optional repeat step 811.The generated image data is then combined to form an isotropic imagevolume 820 or a near-isotropic image volume 822. As will be appreciatedby those of skill in the art, the process of forming one or moreisotropic or near-isotropic image volume can be repeated one or moretimes as shown by optional repeat steps 821, 823.

A method is also described for producing isotropic or near-isotropicimage volume from images. This method is shown in FIG. 8B. The firststep is to obtain an image data volume 830. As shown by optional repeatstep 831, this step can be repeated such that multiple image datavolumes are obtained. Once the image data volume or volumes have beenobtained boundary image data is extracted 840. The extraction processcan be repeated one or more times, as desired, 841. The extracted datavolumes are combined to form isotropic image volumes 842, ornear-isotropic image volumes 844. As will be appreciated by those ofskill in the art, more than one volume can be generated based on theextracted boundary image data 843, 845, if desired.

Another method is provided shown in FIG. 8C. This method includes thestep of obtaining data volume from an image 850, a process that canoptionally be repeated 851, if desired. Once the data volume isobtained, the data volume are combined to form at least one resultantdata volume 860. More than one resultant data volume can be obtained861, if desired. After obtaining the resultant data volume 860, atherapy can be selected based on the data volume 870 or a shape of animplant can be selected or derived 872. Either or both of these stepscan be repeated, if desired, 871, 873.

Referring now to FIG. 7 there is shown an implant design 500 generatedby the methods in accordance with the present invention, whereinthree-dimensional surface has been generated according the methods ofthe present invention. This three dimensional surface can then beutilized to manufacture an implant or select a therapy including animplant. Examples of such implants and implant techniques can be seen inco-pending U.S. patent application Ser. No. 10/724,010, filed on Nov.25, 2003 by Aaron Berez, et al., for “Patient Selectable JointArthroplasty Devices and Surgical Tools Facilitating Increased Accuracy,Speed and Simplicity in Performing Total and Partial JointArthroplasty,” the entirety of which is herein incorporated byreference.

Alternatively, it may be determined after having generated athree-dimensional surface such as that shown in FIG. 7, that this typeof implant is not necessary or cannot be utilized. Thus, a therapy maybe chosen as an alternative or in addition to this type of implant.Examples of therapies include: drug therapy such as pain medication,chondroprotective agents, chondroregenerative agents, bone protectiveagents, bone regenerating agents, bone anabolic agents, bone osteoclastinhibiting agents, injections of hyaluronic acid or chondroitin sulfatesor other drugs or bioactive substances into the joint, osteotomy,chondral or osteochondral autografts or allografts, or other types ofimplants. Other types of implants can include, for example, total orunicompartmental arthroplasty devices.

The methods present herein can be utilized at different timepoints, e.g.a timepoint T1 and a later or earlier timepoint T2. These timepoints canoccur, for example, within one imaging session on a single day, or canoccur over multiple imaging sessions over multiple days. The time spancan further be hours, days, weeks, months and years. Tissues can then becharacterized using quantitative measurements of the resultant datavolumes V1 and V2 and changes in tissue composition or relative orabsolute quantities can be assessed.

The instant invention is shown and described herein in what isconsidered to be the most practical, and preferred embodiments. It isrecognized, however, that departures may be made there from, which arewithin the scope of the invention, and that obvious modifications willoccur to one skilled in the art upon reading this disclosure.

1. A method of improving the resolution of medical images using acomputer processing system comprising: obtaining a first image set of abody part relative to a first plane, the first image set comprising afirst image data volume, the first image data volume having at least afirst planar resolution parallel to the first plane, and a firstperpendicular resolution along an axis perpendicular to the first plane,the first perpendicular resolution being less than the first planarresolution; obtaining a second image set of the body part relative to asecond plane that is different from the first plane, the second imageset comprising a second image data volume; using the computer processingsystem to convert the second image data volume to a common coordinatesystem with the first image data volume; and combining the first andsecond image data volumes to form a resultant image data volume having ahigher image resolution than the image resolution of the first imagedata volume.
 2. The method according to claim 1, wherein the combiningstep comprises: using the computer processing system to obtain from saidfirst and second image data volume first and second gray values at athree-dimensional position; interpolating a resultant gray value fromsaid first and second gray values; and assigning said resultant value toa voxel at said three-dimensional position of said resultant datavolume.
 3. The method according to claim 1, wherein said second plane isat an angle between about 0 and about 180 degrees from the first plane.4. The method of claim 3, wherein the second plane is at an anglebetween about 0 and about 90 degrees from the first plane.
 5. The methodof claim 1, wherein the step of using the computer processing system toconvert the second image data volume to a common coordinate system withthe first image data volume comprises using the computer system toconvert both the first and second image data volumes to a commoncoordinate system.
 6. The method of claim 1, wherein the first image setis taken at a first time and the second image set is taken at a secondtime.
 7. The method of claim 1, further including: selecting a therapyin response to the resultant image data volume.
 8. The method of claim1, further including: selecting a treatment in response to the resultantimage data volume.
 9. The method of claim 1, further including:obtaining at least one additional image set of a body part in a planedifferent than the first and second planes, wherein the additional imageset generates an additional image data volume, wherein the additionaldata volume is converted to a common coordinate system with the firstimage data volume and combined with the first and second image datavolumes to form a resultant data volume.
 10. The method of claim 1,further including: extracting a boundary image data volume from theresulting image data volume.
 11. A method for using a computerprocessing system to produce an improved resolution three-dimensionalimage data of an object comprising: obtaining a first image data volumefrom a first image set of the object taken relatively parallel to afirst plane; obtaining a second image data volume from a second imageset of the object taken relatively parallel to a second plane, thesecond plane being non-parallel to the first plane; using the computerprocessing system to extract boundary image data from each of the firstand second image data volumes; converting the extracted boundary imagedata from the second image data volume to a common coordinate systemwith the extracted boundary image data from the first image data volumeand combining the first and second extracted boundary image data to forma resultant image data volume.
 12. The method of claim 11, furtherincluding: obtaining at least one additional image data volume from atleast one additional image set of the object taken relatively parallelto a plane that is non-parallel to both of the first plane and thesecond plane; extracting an additional boundary image data from theadditional image data volume; converting the additional extractedboundary image data from the additional image data volume to a commoncoordinate system with the extracted boundary image data from the firstimage data volume; and combining the additional boundary image datavolume with the resultant image data volume.
 13. The method of claim 11,wherein the resultant image data volume is near-isotropic in threedimensions.
 14. The method of claim 11, wherein the resultant image datavolume is isotropic in three dimensions.
 15. The method of claim 11,wherein the step of using the computer processing system to convert theextracted boundary image data from the second image data volume to acommon coordinate system with the extracted boundary image data from thefirst image data volume comprises using the computer system to convertboth the first and second extracted boundary image data to a commoncoordinate system.
 16. The method of claim 11, wherein the first planeand second plane are separated by an angle between about 0 and about 180degrees.
 17. The method of claim 16, wherein the angle is between about0 and about 90 degrees.